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ABSTRACT  ] 

A new  series  expansion  method  is  developed  for  a class  of  nonlinear  1 

singularly  perturbed  optimal  regulator  problems.  The  resulting  feedback 
control  is  near-optimal  and  can  stabilize  essentially  nonlinear  systems 

when  linearized  models  provide  no  stability  information.  The  stability  j 
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domain  is  shown  to  include  large  initial  conditions  of  the  fast  variables.  ] 
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The  control  law  is  implemented  in  two-time-scales,  with  the  feedback  from  the  j 

fast  state  variables  depending  on  slow  state  variables  as  parameters.  The  j 

coefficients  of  the  formal  expansions  of  the  optimal  value  function  are  I 

obtained  from  equations  involving  only  the  slow  variables.  I 
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I.  Introduction 

Compared  with  the  rich  literature  on  linear  regulator  theory, 
publications  dealing  with  feedback  design  of  nonlinear  systems  are  a small 
minority.  Realistic  approaches  to  the  difficult  nonlinear  feedback  control 
problem  usually  exploit  properties  of  special  classes  of  systems  to  develop 
approximate  methods  [1,2],  The  approach  in  this  paper  exploits  multiple 
time  scale  properties  of  a class  of  nonlinear  singularly  perturbed  systems  [3,4]  to 
achieve  stabilization  and  near-optimality.  The  stabilization  results  obtained  are 
essentially  nonlinear  in  the  sense  that  they  also  apply  to  the  critical  case 
when  linearized  models  provide  no  stability  information.  Due  to  a separation 
of  time  scales,  the  proposed  design  procedure  is  applicable  to  higher  order 
systems.  ' 

The  problem  considered  is  to  optimally  control  the  nonlinear  system 

X = aj^(x)  + Aj^(x)z  + Bj^(x)u  , x(0)  = x^  (la) 

< 

M-z  = a2(x)  + A2(x)z  + B2(x)u  , z(0)  = z^  (lb) 

with  respect  to  the  performance  index 

00 

[ p (x)  + s ' (x)z  + z 'Q(x)z +u 'R(x)u]  dt  (2) 

0 

where  M-  ^ 0 is  the  small  singular  perturbation  parameter,  x,  z are  n-,m- 
dimensional  states,  respectively,  u is  an  r-dimensional  control  and  the 
prime  denotes  a transpose.  It  is  assumed  that  there  exists  a domain 
D c:  containing  the  origin  such  that  for  all  xeD  and  zeR^*  the  problem 

satisfies  the  following  assumptions: 


f 
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I.  The  functions  aj^,  a^,  Aj^,  ^2’  > ®2’  ^ differentiable 

with  respect  to  x a sufficient  number  of  times  and  a.^,  a2,  P and  s are 
all  zero  only  at  x = 0. 

11.  The  matrices  Q(x)  and  R(x)  are  positive  definite,  that  is,  Q(x)>0, 

R(x)>0.  Furthermore,  the  scalar  function  p+s'z+z'Qz  of  x and  z is 
positive  definite  in  both  x and  z. 

III.  For  every  fixed  xeD 


rankLfi^,  A2B2, 


m 


(3) 


and  hence  A2(x)  is  assumed  to  be  nonsingular.  (If  not,  then  using 
u = u + K(x)z  such  that  A2  + B2K  is  nonsingular  we  redefine  the  problem.) 

Assumptions  I and  II  establish  that  the  origin  is  the  desired  equili- 
brium of  (1).  Assumption  III  and  Q(x)>0  simplify  the  derivations. 
Alternatively  a less  restrictive  stabilizability-detectability  condition 
can  be  used. 

Finite  time  trajectory  optimization  problems  for  the  same  class 
of  systems  have  been  treated  in  [3,4]  via  singularly  perturbed  two  point 
boundary  value  problems  originating  from  necessary  optimality  conditions. 

The  resulting  controls  are  open-loop  and  require  boundary  layer  correction 
terms  at  both  ends  of  the  interval.  For  the  infinite  time  regulator 
problem  considered  here  the  Hamilton-Jacobi-Bellman  sufficiency  condition 
is  more  suitable  since  it  readily  incorporates  stability  requirements  and 
leads  to  feedback  solutions.  Using  this  condition  we  obtain  near-optimal 
stabilizing  controls  in  feedback  form  and  avoid  explicit  treatment  of 
boundary  layer  phenomena. 
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Our  procedure  is  based  on  a nested  power  series  expansion  of  the 
optimal  value  function  in  z and  p.  . An  advantage  of  this  procedure  is  that 

it  uses  lower  order  equations  involving  only  the  slow  variable  x.  In  appli- 

cations truncated  series  are  of  interest.  Stabilizing  properties  of  various 
truncated  designs  are  discussed  and  an  explicit  estimate  of  the  stability 
domain  is  given.  It  is  of  practical  importance  that  this  domain  encompasses 

large  initial  disturbances  of  z(0).  Furthermore,  near-optimality  of  these 

2 

truncated  designs  is  established  in  terms  of  0 (p ) , 0(p  ),  etc.  A particularly 
useful  result  is  that  an  0(p)  near-optimal  feedback  control  can  be  implemented 
without  knowing  the  value  of  the  small  parameter  P. 

The  paper  is  organized  as  follows.  In  Section  II  a reduced  order 

problem  is  formulated  for  the  slow  variable  x.  The  crucial  assumption  is 
that  the  properties  of  its  solution  are  known.  Using  a truncated  expansion 
of  the  optimal  value  function  the  so  called  composite  control  is  introduced 
in  Section  III.  Since  the  leading  term  in  the  series  is  the  optimal  value 
function  of  the  reduced  problem,  the  original  problem  is  well  posed.  In 
Section  IV  it  is  shown  that  the  composite  control  guarantees  a finite  domain 
of  stability  for  the  resulting  feedback  system.  In  Section  V,  a formal 
expansion  of  the  optimal  value  function  is  proposed  and  near-optimality  results 
are  discussed.  An  example  is  discussed  in  Section  VI. 


II.  The  Reduced  Control 

In  singular  perturbation  techniques  [5],  a problem  for  the  full 
order  system  (1)  where  P ^ 0 is  interpreted  as  a perturbation  of  a 
reduced  problem 


X = a^(x)  + Aj(x)z  + Bj(x)u,  x(0)  = x^ 

0 = a2(x)  + A2(x)z  + B2(x)u 


(Aa) 


(4b) 
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in  which  u.=0.  Due  to  Assumption  III,  z can  be  solved  from  (4b)  and  eliminated 
from  (4a)  and  (2).  Then  the  reduced  problem  is  to  optimally  control  the  system 


X = a (x)  + B (x)u  , x(0)  = X 

o o o 


with  respect  to 


where 


J = [ P (x)  + 2s ' (x)u  + u 'R  (x)u] dt 

o o o o 

0 


^o  ^1  " ^1^2  ^2 


B = B - A,a’^B^ 
o 1 12  2 


Po  = P - s'A'^a^  + 


^o  = 22^r'^QA2S  - J 


^ b’a’-Iqa-'b^  . 


(5) 


(6) 


(7) 


The  origin  x = 0 is  the  desired  equilibrium  of  the  optimally  controlled  reduced 
system  (5)  for  all  xeD,  since,  in  view  of  Assumption  II,  a^(0)=0  and 

p^(x) +2s^(x)u +u'r^(x)u  (8) 


is  positive  definite  in  x and  u. 

The  reduced  problem  (5),  (6)  is  considerably  simpler  than  the 
original  problem  (1),  (2)  because  of  the  elimination  of  the  fast  variables 
and  the  reduction  of  the  system  order.  One  of  the  tasks  of  the  singular 
perturbation  analysis  is  to  establish  whether  the  full  problem  is  well 
posed  in  the  sense  that  its  solution  tends  to  the  solution  of  the 
reduced  problem  as  P -•  0.  If  so,  then  the  next  task  is  to  deduce  the 
properties  of  the  original  problem  from  the  properties  of  the  reduced  problem. 


i 
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Finally  these  properties  are  to  serve  as  a basis  for  a simplified  design 
procedure . 

To  formulate  our  basic  assumption  about  the  properties  of  the 
solution  of  the  reduced  problem  we  use  the  optimality  principle 


0 = min[p  (x)  + 2s'(x)u  + u'R^(x)u  + L (a^(x)  + B (x)u)]  (9) 

u 

where  L is  the  optimal  value  function  and  is  its  partial  derivative  with 
respect  to  x.  This  yields  the  minimizing  control 


u = -r"^(s  + ^ b'l') 

o o o 2 o X 


(10) 


whose  elimination  from  ( 9 ) results  in  the  Hamilton-Jacobi  equation 


0 = (p  - s 'R  ^s  ) + L (a 
o o o o'  X o 


B r"^s  ) - 7 L B r'^B'L',  L(0)  = 0. 

ooo'  4x000  X 


(11) 

Note  that  .due  to  (8),  p - s'R  is  positive  definite  in  D.  Our 

o o o o 

crucial  assumption  is  then  stated  as  follows. 


IV.  The  unique  positive  definite  solution  L(x)  of  (11)  exists  in  D and  is 

differentiable  with  respect  to  x a sufficient  number  of  times. 

Furthermore  the  level  surface  L = c = constant  is  taken  to  be  the 

o 

boundary  of  the  set  D. 

In  the  special  case  considered  in  [1],  where  the  linearization  of  (5)  at  x = 0 
is  stabilizable  and  its  states  are  observable  in  the  quadratic  approximation 
of  J^,  our  Assumption  IV  is  automatically  satisfied  for  all  x near  the  origin. 
It  follows  from  Assumption  IV  that  u^  is  the  unique  optimal  feedback  control 
for  the  reduced  problem  and  L is  a Lyapunov  function  of  the  optimally 
controlled  reduced  system 


d, 
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x = a -Br''(s  B'l'  ) = a (x) 
o oo  o 2 Ox  o'' 


establishing  that  the  origin  is  asymptotically  stable  and  the  set  D belongs 
to  its  domain  of  attraction. 


III.  The  Composite  Control 


The  optimal  value  function  V(x,z,p.)  of  the  full  problem  (1), 


(2)  satisfies  the  equation 


0 = min[p  + s'z  + z'Qz  + u'Ru  + V^^Ca^^  + Aj^z  + Bj^u)  + 


- + AjZ  + BjU)] 


where  V , V denote  the  partial  derivatives  of  V with  respect  to  the 
X z 

variables  x,  z,  respectively.  The  minimizing  control  of  (13)  is 

u - - i R-hBjv;  + i B^v;) 

and  its  substitution  into  (13)  yields  the  Hamilton- Jacobi  equation 


0 = p + s'z  + z'Qz  + V^(a^  + A^z)  + - + k^z) 

- 4<vi  S’  + 1 ’■2^'’  • “ • 


Since  system  (1)  is  linear  in  z and  J in  (2)  is  quadratic  in  z, 
and  since  z is  multiplied  by  u , we  seek  a solution  of  (15)  in  the  form 


where 


Vfx.Zjk)  = Vq  (x) +M.  v|  (x)z +4  z ' V2  (x)z +Kq  (x,z  .►i  ) 
= V(x,z,M.) +i-.q(x,z,M.)  , Vq(0)=0 


dq/Sx  = 0(1),  dq/dz  = 0^). 


_!  t 
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We  shall  investigate  the  expansion  of  q in  a later  section.  The  partial 
derivatives  of  V with  respect  to  x,z  are 


V^,  = V^^+0^) 

V = nv!  +2pz'v„  +0(h.'^)  . 


M8) 


Substituting  (18)  into  (15)  and  neglecting  the  u dependent  terras,  we  obtain 
the  equation 


0 - p+v„^a,+v;a,  -i-  +V;ii2)R-hli;v'^+»^V,) 

+ "s'  +2a^V^  ^'^2  ' ^2^' ' *^2^2  ^ 

+ z'(Q  + V2A  i-A2V.,-V2B2il'^h.;V2)z. 

in  order  to  sati.;-.i\  (i''')  iceiiticd'K  rol  all  z,  w>  inquire  that 


i -t; 


0 = p +VQ^a,  1 '.y,  - (\yi5j  +V^112)R  (Si'y  ■:  iCi’i).  vyo) 


= 0 


0 = s ' +2a2V.,  + (Aj -BjR'S..^V,)  + vj  (A2-B,,R"^B,' V2) 


0 = Q + V2A2 +A2V2-V2B2R'  B2V2  . 


ll'O 


120) 

(21) 

(221 


At  each  fixed  value  of  x,  (22)  is  an  algebraic  Riccati  equation  for  In 

view  of  (3)  and  C(x)1^0,  the  unique  positive  definite  solution  exists  such 
that  for  all  xeD,  the  real  parts  of  the  eigenvalues  of  A2=A.,-B2R  ^82^2, 
denoted  by  Rell(A-,)j,  arc  less  than  a negative  constant.  Thus  k,y  is  non- 
singular and  can  oe  expressed  in  terms  of  and  as 


= - s ' +2a,,V2  + Vy, 


-1  I-  ---1 

(A,-B  R B,,V,,  ) At 


(23) 


It  is  of  crucial  i nip  r ranee  that  the  elintination  of  Vj  from  (21)  results  in 
an  equation  involving  only  Vq^  . J’or  the  well  posedness  of  tlie  full  problora 
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it  is  necessary  that  the  leading  term  Vq  of  (16)  be  identical  to  the  solution 
L of  the  reduced  problem. 

Lemma  1 


If  Assumptions  III  and  IV  are  satisfied,  then  the  unique  positive 
definite  solution  Vq(x)  of  (20)-(22)  exists  in  D and  is  identical  to  the 
solution  L(x)  of  the  reduced  problem  (5),  (6). 

Proof : It  is  shown  in  the  Appendix  that  eliminating  Vj^  from  (20),  we  obtain 
the  Hami Iton- Jacobi  equation  (11)  with  V in  place  of  L^,  and  hence 
Vq(x)  = L(x)  with  properties  as  in  Assumption  IV. 

By  virtue  of  Lemma  1,  Vq  and  V^  are  solved  independently  from  (11) 
and  (22).  This  is  the  separation  of  time  scales  in  the  design  of  nonlinear 
regulators,  analogous  to  the  linear  time-invariant  design  in  [7]. 

Using  V,  we  derive  the  control 


u = - |R-'(B;v;+iBX) 

= - J r‘^[b{Vq^ +B2(V^ +2V2Z)]  +0(p) 

= u +0(ii) 
c 

whose  main  part  u^  is  defined  as  the  composite  control. 
(24)  using  (23)  and  following  the  derivation  in  [7], 


(24) 


Eliminating  Vj^  from 

u can  be  written  as 
c 


"c  ■ ■''o‘<“o+2  “X')! 


where  _ ^ _ 

A2(x)  = A2-B2R’  B2V2 

a2(x)  = a^-j 


(25) 

(26a) 

(26b) 


Hence  the  composite  control  u consists  of  a slow  control  u which  optimizes 

c o 

the  reduced  system  (5)  and  a fast  control  -R  ^82^2  (z+A  ^32)  which  optimizes 

the  fast  part  (z +A  ^3^2)  ^ the  sense  that  satisfies  (22).  Note  that 

when  z is  not  penalized  in  (2),  that  is  when  Q(x)  = 0,  but  Re[>.(A2)}  0,  then 

V2  is  identically  zero  and  u^  reduces  to  u^  of  (10).  Stabilizing  properties 

of  the  composite  control  u are  established  in  the  next  section. 

c 


IV.  Stabilizing  Properties 


System  (1)  controlled  bv  u is 

c 


X = a.  +A,z  +B,u  = 

1 1 1 c 

a^^  (x)  +Aj^  (x)z. 

x(0)  =x 

= a_  +A^2  +B„u  = 
2 2 2 c 

32  (x)  +A2  (x)z, 

z (0)  = z 

0 

where 

^1  " ^1  ’2  ^l^'^'^^l^Ox  3^(0)  =0 

A^  = A^-B^r‘^B2V2. 

With  the  change  of  variables 

P = z+A2^a2 

exhibiting  Ti  as  the  fast  part  of  z,  system  (27)  becomes 


(27) 


(28) 


(29) 


^ ^ ®o ‘''^1^'  ’ x(0)=x^  (30a) 

(A^  ^2)^3^  +[A2  +11  (A2^a2)^A^]  ■ 

= Hf(x) +[A2(x) +pF(x)]Tl  , T1(0)  = z^ +A^^(x^)a2(x^).  (30b) 

Since  the  right-hand  side  of  (30b)  is  an  0(}i)  perturbation  of  ^ (x)I  and 
Re{X(A2)]<0  in  D we  expect  that  T)  will  rapidly  decay  to  an  0(fi)  quantity. 


This  motivates  the  introduction  of 
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U(x,-",e)  = Vq(x) +?’ 'V2(x)"'  (31) 

as  a tentative  Lyapunov  function  for  (30).  Here  C is  a small  positive  scalar 
to  be  determined.  From  Assumptions  III  and  IV,  Vq(x)  is  positive  definite 
and  V2(x)>0  in  D.  Hence  U is  positive  definite  for  all  xeD  and  T'cr'". 
Furthermore,  since  Vg(x)=c^>0  for  all  x on  the  boundary  of  D,  the  surface 

S(x,T',,e)  = {x,r,  : U(x,T1,e)  =c^]  (32) 

is  closed  in  the  (n+m) -dimensional  domain  xeD,  'PeR'''.  We  define  S.  to  be 
the  domain  in  the  interior  of  S. 

Let  be  a set  strictly  in  the  interior  of  D,  that  is,  the  boundary 
of  does  not  intersect  the  boundary  of  D,  and  let  E be  a bounded  set  in  R™. 
The  presence  of  £ in  U extends  S to  encompass  (x,t;)  for  all  xgD^  and  for  in 

any  prescribed  set  E.  This  crucial  result  is  stated  as  follows. 

Lemma  2 

If  Assumptions  III  and  IV  are  satisfied,  then  there  exists  an  £ ^ 0 

such  that  the  domain  S.  contains  all  xeD,,  TgE. 

in  1 

Proof:  At  each  point  xeDj^,  the  projection  S onto  the  T]  subspace  is  the 
e llipsoid 

T1'V2(x)T',  = (c^-VQ(x))/e  (33) 

implying  that  11  extends  to  0(l/v^).  Hence  for  every  x,  there  exists  an  £(x) 
sufficiently  small  such  that  the  ellipsoid  (33)  includes  all  ^ieE.  (Note  that 
we  must  exclude  the  boundary  of  D because  from  (33)  the  projection  of  S at 

any  point  on  the  boundary  of  D is  a single  point  T1  = 0.)  Hence  choosing  £*  to 

be  the  smallest  of  such  £(x),  the  domain  S.  contains  all  xeD,,  "^eE  for  any 
ee(o,e*l . 
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By  virtue  of  Lemma  2,  the  initial  condition  11(0)  of  (30b),  and 
hence  z(0)  of  (27),  can  be  as  far  away  from  zero  as  0(l/'y^)  and  still  be 
enclosed  by  S.  We  now  examine  the  relationship  between  t and  p . 

Using  (11),  (22)  and  rearranging,  we  obtain  the  time  derivative  of 
U with  respect  to  (30)  as 


ii  = -g(x,e,u)  e'Q(x)?  T’M(x,-,e,n)ri  (34) 


where 


I -1 

g = y Q y 

8i  = Po-®X^®o+4  VVo^^Xx 

y = AjVQ^ +2eV2f  (35) 

§ = 'T'  q‘V 

M = ^ + V2B2R'^B2V2-^  (V2F +f'V2)-M.  V^. 

Since  V2F  + f'V^  and  V2  are  bounded  for  all  x,"  in  and  since  Q(x)>0 

in  D,  it  follows  that  there  exists  a such  that  M>0  for  all  x,l'  in  S. 

1 in 

and  for  ^€(0,^.'^'!.  Thus  the  last  two  terms  in  U are  positive  definite.  To 
ensure  that  g(x,F,,M.)  is  positive  definite,  we  assume  that  the  reduced  problem 
also  satisfies 
V.  The  limit 


■1 


lim  ^ = k(e)<t 


|xh0  ®1 

exists  for  all  fixed  fi>0. 


(36) 


Note  that  k>0  because  y'Q  ^y  is  positive  semidefinite  and  g^^  is  positive 
definite.  The  limit  (36)  implies  that  there  exists  a domain  D about  x = 0 
such  that 

y'Q’^  < (l-Hc)gj 


(37) 
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that  is  such  that  for  u.<2e/(l+k),  g is  positive  definite  in  D,  see  (35). 
Let  k(?)>0  be  the  minimum  value  of  on  the  boundary  of  D.  Hence  in  the 
domain 


5^  (x)  = {x  ; g^  (x)  <'ir}  (38) 

g is  positive  definite.  On  the  other  hand,  since  D is  bounded,  there  exists 
a kj(;^)>0  such  tliat  y'Q  ^y  <"  k^  for  all  xeD,  that  is  such  that  g is  positive 
definite  when  x is  not  in  the  domain 


D(x)  = {x  : gj^  (x)  <uk^/2e] 


(39) 


about  the  origin.  But  for  M.^28k/kj^,  DC  5^ , implying  that  g is  positive 
definite  in  D.  Thus  U is  negative  definite  for  all  x,"!]  contained  in 
We  now  conclude  that  U is  a Lyapunov  function  for  (30)  guaranteeing  that 
x = 0,  ^ = 0 is  asymptotically  stable  for  all  xeD^^,  and  for  pe(0,p,*l,  where 


■ .28  2ek 


(40) 


Returning  from  the  " variable  to  the  z variable  via  a 2 , 

we  obtain  for  all  xeDj^,  '^eE  a corresponding  bounded  domain  E^^  for  z.  We 
summarize  the  above  discussions  on  the  asymptotic  stabilizing  property  of 
u^  in  (24)  as  follows. 

Theorem  1 


I f Assumptions  I-v  are  satisfied,  then  there  exists  a (J.*  > 0 such 
that  for  all  M.e(0,p.*3  and  for  all  xeDj  and  z in  any  prescribed  bounded  set 
Ej^,  the  origin  x=0,  z = 0 of  the  feedback  system  (1)  controlled  by  the 
composite  control  u^  is  asymptotically  stable. 

Theorem  1 can  be  applied  in  two  different  directions.  As  outlined 
above,  for  any  given  and  Ej^ , we  first  find  ?*  such  that  of  (32) 


contains  all  xeD^^,  zeEj^.  Then  we  find  ii*  from  (40).  This  direction  is 
suitable  when  p.  is  a parameter  at  the  designer's  disposal,  such  as  a gain 
factor  [9] . In  the  other  direction,  if  p represents  some  given  physical 
parameters,  such  as  time  constants,  we  use  its  value  to  determine  the 
smallest  G such  tliat  U of  (34)  is  negative  definite,  that  is  we  find  the 


largest  and  Ej . 

As  a special  case  of  Assumption  V,  consider  that  the  origin  x = 0 
of  the  reduced  system  (12)  is  exponentially  stable.  Then  near  the  origin, 


■1 


p -s'R  s , Vr,  grow  as 
o o o o’  “ 


Q ao  |x|  , and  |Vq^|,  |a^| 

positive  constants  and  6 such  that 


grow  as  |xl  , and  w»e  can  find 


^21x1 


< p -s  'R  ^s  < k [ x|  ^ 
— o o o o — 3 ' 


I x!  ^ < Vq  < k^l  xI  ^ 


< IVq^I  1 k^lxl 


(41) 


for  all  |x 
such  that 


|x|  "6. 


1 Uol  C kgU! 


It  follows  from  (41)  that  there  exists  a fixed  kjQ(G)^0 


y'Q'^  < ^^qIx 


(42) 


and  the  limit  (36)  is  bounded  by 


lim  ^ ^ ^ ^ lim 

Ixl-O  ®1 


kiol X, 
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'xl-0  k.,|x|^ 


(43) 


satisfying  Assumption  V. 


In  this  case  a claim  stronger  than  Theorem  1 can  be  made. 
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Corollary  1 

If  Assumptions  I- IV  are  satisfied  and  the  origin  x = 0 of  the 
reduced  system  is  exponentially  stable,  then  the  conclusion  of  Theorem  1 holds 
and  moreover  the  origin  x = 0,  z = 0 of  (27)  is  exponentially  stable. 

Proof : The  first  part  of  the  corollary  follows  from  Theorem  1.  The  second 

part  follows  from  the  linearization  of  (27)  at  the  origin 


6x 

^ Sa^(O) 
dx 

Ai(0) 

6x 

62 

1 ^^2^°)  1 

4 Sx  4 

A2(0) 

6z  1 

(44) 


The  system  matrix  of  (44)  has  one  group  of  n small  eigenvalues  0(p)  close 

Saj^ 3a2j 

to  those  of  ^-A^A2 


and  another  group  of  m large  eigenvalues  0(1) 
x=0  _ 1 Sa, 


1 ^ **  ]_  Q 

close  to  those  of  — A„(0)  [8]  . But  a, -A, A.,  a „ = a and 
aa,  i i 2 ^ 


1 ^ ^-1  ''21 

'x=0 


ax  ■ ■^1^2 


jJ.  ' i I T!  Z O ax  ' x=0 

as  a„(0)  = 0.  Thus  the  real  parts  of  the  eigenvalues  of 


the  system  matrix  of  (44)  are  all  negative  and  x=0,  z = 0 is  exponentially 
stable . 


If  the  origin  x = 0 of  the  reduced  system  is  only  asymptotically 
stable  but  not  exponentially  stable,  then  in  general  g need  not  be  positive 
definite  for  all  xeD.  This  situation  includes  the  critical  case  when  the 
linearized  model  does  not  provide  any  stability  information  as  clarified  by 
the  example  in  Section  VI.  For  this  situation  the  system  is  now  shown  to 
possess  a weaker  stability  property,  that  is,  its  trajectories  tend  to  a 
small  sphere  around  the  origin.  Define  the  domain  in 


P (x)  = [x  : g(x,e  ,M. ) < O] 


(45) 


which  is  contained  in  the  domain  D of  (39).  Due  to  the  presence  of  p.  in 


(34),  U may  be  positive  only  if  xep  (x)  and  Tl=0(ii).  Otherwise,  U is 
negative.  Defining  the  surface 
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TT(x,z)  = {x,z  :xep(x;p),  z - -A2^(x)a2  (x)] 

about  the  origin  in  u^  defined  by  (24)  is  a stabilizing  control 

following  sense. 

Theorem  2 


(46) 
in  the 


If  Assumptions  I-IV  are  satisfied,  then  there  exists  a M.*>0  such 
that  for  all  M'e(0,p.*],  the  feedback  control  (24)  steers  all  xeDj^,  zeEj^  of  the 
full  system  0(m,)  close  to  the  surface  Tr(x,z). 

Proof : Since  U>0  and  ll<0  except  for  xep  (x)  and  r = 0(M,),  x converges  to 
p (x)  and  T decays  to  an  O(M')  quantity.  Thus  in  the  x,z  variables , /'x,zj  converges 
to  an  0(|i)  neighborhood  of  the  surface  'n'(x,z). 

In  the  case  where  the  fast  transients  of  z in  (1)  are  exponentially 
stable,  that  is,  A2(x)  is  stable  for  all  xeD,  and  we  are  only  concerned  with 
the  optimality  of  the  reduced  system  (5),  then  the  z-independent  reduced 
control  u^  of  (10)  stabilizes  the  full  system  (1)  with  essentially  the  same 
stabilizing  properties  as  u^  of  (24).  We  shall  not  repeat  the  argument. 

An  attractive  feature  of  the  controls  u and  u is  that  they  do 
not  require  the  knowledge  of  the  actual  value  of  p.  provided  that  it  is 
sufficiently  small.  When  appropriately  implemented,  these  controls  stabilize 
the  full  system  (1)  and  achieve  optimality  of  the  reduced  system,  and  in  the 
case  of  u^,  also  optimality  of  the  fast  part  of  z.  The  above  results  also 
answer  the  question  of  well  posedness  by  giving  the  conditions  under  which 
the  same  optimal  reduced  order  system  is  obtained  when  is  set  equal  to 
zero  either  when  system  (1)  is  uncontrolled  or  when  it  is  controlled  by 
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u or  u . In  contrast  to  many  other  singular  perturbation  results  which 
c o 

require  p to  be  sufficiently  small,  this  section  provides  a method  to 
compute  an  estimate  of  allowable  values  of  p given  a stability  domain  or 
vice  versa. 


V.  A Formal  Expansion  and  Near-Optimality 
The  expansion  (16)  only  satisfies  the  Hamilton- Jacobi  equation  (15) 
to  0(m,)  order.  We  now  propose  to  solve  (15)  by  expanding  V formally  as  a 
nested  infinite  power  series.  If  this  power  series  is  convergent,  then 
the  optimal  solution  V of  (15)  exists.  For  x,z  near  the  origin,  it  has  been 
shown  in  [ 1]  that  the  optimal  solution  exists  and  possesses  a power  series 
expansion  when  system  (1)  after  linearization  at  the  origin  is  stabilizable 
and  the  state  in  the  quadratic  approximation  of  J is  observable.  Here  we 
are  interested  in  a power  series  of  V which  satisfies  (15)  to  any  order  of  p. 

Since  system  (1)  is  linear  in  z and  J is  quadratic  in  z,  the 
optimal  value  function  can  be  expanded  as  a power  series  in  the  components 
of  z [2] . In  addition,  since  z is  the  fast  variable,  the  z terms  in  the 
optimal  value  function  are  multiplied  by  appropriate  powers  of  p.  [5] . In 

view  of  these  two  characteristics,  we  seek  a solution  of  (15)  in  the  form 

m mm 


V(x,z,M-)  = V«(x,M-) .1  V,  . (x,M-)z  +H.  Z Z V , (x,M-)z  z, 
^ J = 1 J-J  J j = l k-1  J K 

2 m m m 

+ P 2 E Ev.  (x,p)z.z,  2 +••• 

j = l k=l  q=l  2jkq^  J k q 


+ ^ 


i-1 


m 

E 

ii=i 


m 

Z ■ 


m 

■ ^ V., 
j =1 


1 ■‘2 


i 


I 


Vq(0,p)  =0 


(47) 
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where  V. . . .is  the  (j, , • • • j j • ) element  of  the  completely  symmetric 
1 ^ 1 .j. 

generalized  matrix  of  dimension  m and  is  the  jth  component  of  z. 

The  summation  signs  in  (47)  and  in  other  equations  in  the  paper  will  be 

omitted  when  there  is  no  confusion  as  to  which  indices  j , ^2  > • • • > 

are  being  summed.  The  partial  derivatives  V ,V  expressed  in  terms 

1 m 

of  the  vector  x and  the  scalars  z. ,...,z  are 

1 m 


''x  = + 


(48a) 


\ + 2^V2,.e.  + 3^.  V3.  .j^z.zj^  + i=1.2.....m  (48b) 

where  the  summation  signs  over  3 ,k  are  omitted. 

For  the  series  (47)  to  satisfy  (15)  as  an  identity,  we  first 

rewrite  (15)  in  terms  of  the  vector  x and  the  scalars  z,,.,.,z  , 

1’  m 


0 - p+s.z^  +il 

- t <Vi  + k * P 


(49) 


where  s^,  a2^  are  the  ith  components  of  the  vectors  s,  a2,  respectively, 
is  the  ith  column  of  the  matrix  Aj^,  82^^  is  the  ith  row  of  B2,  Q2ij> 

A2ij  (ijj)  elements  of  Q,  A2>  respectively,  and  the  summation  signs 

over  the  indices  i,j  are  omitted.  Then,  upon  substituting  (48)  into  (49) 
and  equating  the  coefficients  of  the  like  powers  of  z^,  we  obtain 


The  (j^*  j2  > • * • » Jf)  elements  of  are  identical  for  all  permuta- 
tions of  the  Indices  j2»  • • • » [6]. 
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0 . P + V^a^+Vj^a^j-i  ( , )r-\b;v^  + B' . Vj^) , 


Vq(0,M-)  =0 


(50a) 


^ ■^^'^2ij®2j  ■ 2 

+ V,.B,)r-'(pb;v;^^+2B-,V2,P  , 1-1,2, ...,« 


0 ■ Olj-"‘‘''21jR‘‘l*'*<''ua''lj>s+2(V2iRA2Rj),+3aV3,.^a2^ 

-J  <''  a'‘l+''lk»2k>'''‘<‘‘“l''2Ijx+^‘‘“ik''3klj> 

-Z  *‘''lix8l+2''2ik“2k)''''«'»l''ijx+^»k''2kj>  > 

ijj  ~ l}2j*«*jTn 

" ■ '‘^''31Jkx'‘l-""-‘''2IJx*lk>s+'*-^''41jkq*2q+^<y31jq*2qk)s 

- I (''Ox»l  + ''lq’‘2,)'>'  ' jkx  ^ '“''»2q''4i  jkq> 

-J  <"''llx“l+2''21q“2q>'*''‘"“;''2jkx+^“2q''3qjk))s’ 

IjJjk  lj2j***jni 


(50b) 


(50c) 


(50d)^ 


where  the  right  hand  sides  of  (50a) , ^0 b) , (50c) , (50d) , . , . , are  the 
coefficients  of  the  z-independent  terms  and  of  the  z.,  z.z.,  z.z.z 

1 1 j 1 j K 

terms,  respectively.  Because  of  symmetry,  there  are  m(m+l)/2  equations 

i-1 

in  (50c),  m (m+1 ) (m+2 ) / 6 equations  in  (50d)  and  in  general,  (^n^ (m+k ) ) / i I 
equations  when  the  coefficients  of  z.  z.  ...z,  , =1,2, ...,m, 

Jl  J2  Jf 

are  equated. 


The  subscript  s denotes  the  symmetrization  operation  of  generalized 
matrices  [6] . For  example, 

<''2Ik''2kJ>s-?  <''2ik*2kJ +''21k''2kl> 

3ijq  2qk^s  6^  3ijq^2qk  '^3jiq^2qk^'^3ikq^2qj^'^3kiq^2qj^'^3jkq^2qi^'^3kjq^2qi^  ‘ 
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For  a simplified  treatment  of  these  equations  we  now  exploit  the 
presence  of  the  small  singular  perturbation  parameter  p. . We  expand  each 
coefficient  of  (47)  as  a power  series  in  p 


V (x,p)  = .2  ^l^V^(x)  , i = 0,l,2,.., 

1 j=0  1 


(51) 


where  the  boundary  condition  of  is  Vq(  0 ) = 0,  j =0,1,2,...  The 
expressions  (51)  substituted  into  equations  (50)  are  to  satisfy  them  as 
identities  in  p.  Equating  the  coefficients  of  the  like  powers  in  p,  we 
generate  sets  of  equations  for  , i, j = 0, 1 ,2 , . . . The  first  set  of  equations 
obtained  by  equating  the  P -independent  parts  in  (50a),  (50b),  (50c), 
are  precisely  equations  (20),  (21),  (22),  respectively.  Hence  from  the 
uniqueness  of  solutions  to  (20),  (21),  (22).  We  conclude  that 


.0  - 


,.0 


.0  TT 


Vo  = Vo  = L,  V^  = Vj,  V^  = V^ 


and  V thus  consists  of  the  leading  terms  of  V. 

The  second  set  of  equations  in  matrix  form 


(52) 


“ • ''w“i  * “ 

0 . + aP?:  + v!'a,  + 


I'lx  ■ '1  "2  ““2 '2 

.0  - . 1/..0  r . r'..0'.  . ..Ir  . -...1 


0 - + v‘a2  + A^v‘  + 3(v»r2) 


0 - 3(V3%),  + 


(53a) 

(53b) 

(53c) 


(53d) 


( 
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obtained  by  equating  the  terms  in  (50a),  (50b),  (50c),  (50d),  respectively, 
involve  only  the  unknown  terms  , Vj^ , V2  and  V^.  In  (53)  the  multi- 
plication of  an  n^^xn^Xn^  matrix  by  an  n^Xn^  matrix  results  in  an  n^Xn^Xn^ 
matrix.  For  convenience  we  suppress  the  last  dimension  of  the  mXmXl 
matrices  and  (V2a2)  regard  them  as  mXm  matrices.  Since  A2  is 

stable,  (53d)  and  (53c)  can  be  solved  sequentially  for  and  V^,  respectively. 
Then  v|  can  be  solved  from  (53b)  and  its  substitution  into  (53a)  results  in 
the  partial  differential  equation 

In  general,  in  equating  the  ^ ^ terms  we  obtain  the  (i+l)st  set 
of  equations  involving  the  unknown  terms  Vq^,  \ . . . , V?_|^2  • The 

terms  > • • • > ^2  ^ ^re  solved  for  sequentially  and  then  Vq  ^ is  to  be 

solved  from  an  equation  similar  to  (41). 

The  main  accomplishment  of  the  nested  expansions  is  that  the  first 
set  of  equations  (20)- (22)  can  be  solved  independently  for  the  first  three 
zeroth  order  terms  Vq,  V^,  and  V^-  Similarly,  (53)  and  the  subsequent  sets 
of  equations  can  be  solved  independently  for  Vq,  , . . . , V?_^2  • These  equations 
are  dependent  only  on  x and  not  on  z or  M'  . A further  simplifying  property 
is  that  at  the  first  stage  the  equations  (11),  (22)  for  Vq  and  are 
decoupled . 

The  approximation  obtained  by  expanding  V of  (47),  (51)  to  the  ith 
set  of  equations  is  stated  in  the  following  theorem. 

Theorem  3 

Suppose  that  the  solutions  to  the  ith  set  of  equations  of  V exist 
and  let  be  the  truncated  series  of  (47),  (51)  including  all  the  terms 


4 


I 


I 

I 
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up  to  the  ith  set.  Then  the  control 

1 

1 . 1 i 


“i  ■ - ? « > 


(55) 


is  near  optimal  in  the  sense  that  V satisfies  the  Hamilton- Jacobi  equation 
(15)  to  an  0(|J.^)  error. 

Proof : Substituting  the  terms  into  (15)  and  using  the  first  i set  of 

equations  of  V,  the  coefficients  of  p.  terms,  i,  in  the  resulting 
equation  vanish,  implying  O(p^)  near-optimality. 

Thus  Theorem  3 implies  that  u^  of  (24)  is  an  0(p)  near-optimal 
control  because  it  is  an  0(p)  approximation  of  Uj^  which  achieves  0(u)  near- 
optimality. In  general,  retaining  only  the  terms,  k<i,  in  u^ , the 
resulting  control  also  is  O(p^)  near-optimal  in  the  sense  of  Theorem  3. 

Repeating  the  derivation  in  Section  IV,  we  can  show  that  u^^ 
stabilizes  the  full  system  (1)  with  similar  stabilizing  properties  as  u^  of 
(24).  We  first  introduce  the  x,  '^  = z+A2^a2  variables  and  consider  U in 
(31)  as  a tentative  Lyapunov  function.  The  analysis  is  more  cumbersome 
but  results  similar  to  Theorems  1 and  2 and  Corollary  1 can  be  established. 


VI.  Discussion  and  Example 

The  computational  advantage  of  the  proposed  procedure  is  that  all 
the  terms  of  V in  (47),  (51)  are  obtained  from  equations  involving  the  slow 
variable  x only.  Moreover  V^  and  V^  are  solved  for  independently.  Explicit 
consideration  of  the  initial  boundary  layer  is  avoided  and  it  is  optimally 
stabilized  by  the  z variable  feedback.  Furthermore  using  the  x,Tl  variables 
an  estimate  of  the  domain  of  stability  is  easily  obtained.  Alternatively, 
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for  a stability  domain  to  encompass  a prescribed  bounded  set  r™ 

a bound  for  p.  can  be  determined. 

Several  aspects  of  the  design  procedure  and  the  stability  properties 
of  the  resulting  feedback  system  are  now  illustrated  by  considering  the 
optimal  control  problem  of  the  second  order  system 


X = xz 
p,  z = -z  + u 


(56) 


with  respect  to  the  performance  index 

J = r (x^  +2  J (57) 

0 

0 2 2 

Solving  the  reduced  problem  we  obtain  L = Vq = x and  u^  = -x  . 

3 

The  optimally  controlled  reduced  system  (12)  is  x = -x  and  its  unique 
asymptotically  stable  equilibrium  is  x=0.  Note  that  the  linearization 
of  the  reduced  system  fails  to  provide  any  stability  information  at  x = 0. 
Let  D be  th  i interval  (-1,13,  that  is,  L=c^=l  at  x = +1  by  Assumption  IV. 

The  pair  (A2 , 82 ) = (- 1 , 1)  satisfies  (3)  and  we  can  solve  (22)  for 
V2  = j (vT-l)  such  that  A2  = -V2.  Then  the  substitution  of  Vq  = L = x^  and 
V2  into  (23)  yields  the  following  expressions  for  (24)  and  (16) 

u^  = + (c/2-1)  z)  (58) 

V = x^ +p.V2x^z +P’  j (V2-l)z^.  (59) 

The  resulting  feedback  system  is 


X = xz 

p z = - v^x^  - v'2'z  , 


(60) 
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This  result  is  essentially  nonlinear  since  the  linearization  of 

(60)  at  x = 0,  z = 0 does  not  provide  any  stability  information.  Using  the 

2 

change  of  variables  = z+x  , system  (60)  becr>mes 


• - 2 , . 

X = -X  +x 

lit  = -2|J.x^  - (\^-2m.x^  )T| . 


(61) 


Since  we  require  |x|  ^1,  p.  is  restricted  to  be  less  than  l/’Jl.  The 
tentative  Lyapunov  function  (31)  is 


U(x,Tl,e)  = (^/2-l)e.Tl^.  (62) 

If  we  require  that  the  initial  conditions  of  (61)  be  in  jxl  < .8,  ^5, 

then  we  must  set  C to  be  less  than  .0695  in  order  for  the  ellipse 


S(x,T1,R)  = {x.Tl  : U = x^ +j  (^/2-l)e1^^  = l} 


(63) 


to  enclose  these  initial  conditions.  Plots  of  S in  the  x,!!  coordinates 
and  the  x,z  coordinates  for  £=  .06  are  shown  in  Figure  1.  The  time 
derivative  of  U with  respect  to  (61)  is 


where 


g^  = 2x^,  y = 2(1  -£(v/2-l)x^)x^ 

5 =''1-^y,  M = |- - 2p(v/2-l)x^  . 


(64) 

(65) 
(65) 


Since  lim  y /g  =2,  Assumption  V is  satisfied.  For  all  x.T)  in  the  interior 
x^O  ^ 

of  S and  £ = . 06 , U is  negative  definite  for  all  p e (0, • 033 . Hence  x = 0 , z = 0 
is  asymptotically  stable  for  all  |x|<.8,  | z +x^|  < 5 and  pe  (0, .031 . 
Furthermore,  V satisfies  the  Hami Iton- Jacobi  equation  (15)  with  an  error 
of  p2  vT x^z^  . 
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If  we  are  only  interested  in  the  optimality  of  the  reduced  problem 

and  consider  the  z-part  as  due  to  "system  parasitics,"  we  can  apply  the 

reduced  control  u^  to  (56)  as  A2  “ "1  is  stable.  System  (56)  controlled  by 

u is 
o 


X = xz 

2 

p.  Z = -X  - z . 

2 

Transforming  z to  T]  = z +x  , system  (66)  becomes 


(66) 


X = -x^  +xTl 

\ii  = -2tJ.x^  - (1  - 2px^)'"  . 


(67) 


We  use  U in  (62)  as  a Lyapunov  function  for  (67)  and  the  time  derivative  of 
U with  respect  to  (67)  is 

U = -[2  2('v/2-l)(v/2+l  -ex^)^1x'^  ■£  2(v^+l  -£x^)x^] 

- ^ (.2-l)(|--2px^)T',^  (68) 

Thus  for  all  x,Tl  enclosed  in  S and  e=  .06,  U is  negative  definite  for  all 

pe(0,.02].  Hence  x = 0,  z = 0 of  (66)  is  asymptotically  stable  for  all 

(xl<.8,  |z-hc^|<5,  P€(0,.02]. 

2 

To  obtain  an  0(p  ) approximation  of  V in  the  sense  of  Theorem  3, 
we  solve  (53)  for  higher  order  terms  of  vj  and  obtain 

U2  = u^  - p2x^z  (69) 

= V+p  — +^i2^2^2_  ^20) 

System  (56)  controlled  by  U2  becomes 
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X = xz 

p,z  = -V2x  - (-72 +n2x^ )z , (71) 


or,  in  the  x,''  = z+x  variables, 


X = -x^  +xTl 
uTi  = -V^T] 


(72) 


which  is  globally  asymptotically  stable  for  all  Furthermore,  V‘ 

2 ^2  2 3 

satisfies  (15)  with  an  error  of  |i  (8x  z +2x  z ). 


VII.  Conclusions 

A nested  power  series  expansion  method  has  been  proposed  for 
solving  the  optimal  control  problem  of  a class  of  nonlinear  singularly 
perturbed  systems.  The  terms  in  the  expansion  V are  obtained  from  equations 
involving  only  the  slow  variable  x.  In  addition,  and  are  solved  for 
independently.  Explicit  consideration  of  the  initial  boundary  layer  is 
avoided  and  it  is  optimized  by  the  z variable  feedback.  Sufficient  condi- 
tions are  obtained  such  that  feedback  controls  using  truncated  series 
stabilize  the  nonlinear  systems  and  the  stability  domain  can  encompass  large 

initial  conditions  of  z.  These  truncated  controls  can  achieve  near- 

2 

optimality  of  0(m.),  0(p.  ),  etc.  In  particular,  an  0(y,)  near-optimal  feed- 
back control  can  be  implemented  without  knowing  the  value  of  the  small 
parameter  p . The  results  apply  to  essentially  nonlinear  problems. 


Appendix 


Substituting  (23)  into  (20)  and  rearranging  yields 


where 

= P - (s'  +2a^V2)A2‘^a2  - s'  +a^V2)52^B2R‘^B^A2'^|-  s +V2a2) 

''2  = 

X„  = B r’^b' 

3 o o 

•o  ■ “1  - 

“o  * “1  - (Ai-BiR''b'V2)A2‘b2 

A2  = A2  - B2*^'^®2'^2 

and  the  superscript  0 in  vJx  and  V2  has  been  dropped.  Let  H = I +R  ^B^ 

Then  h’^  = I - r'^B  ' V„a'^B„  and  h''^RH'^  = R + BJA  ' '’"QA'^B.  = R . Thus 
2/22  22  22  o 

B = = B H.  Hence  = B R ^B  * . Also, 

0II220  3ooo  ’ 

*2  ■ (R+B2'^'V'*B2)R-‘b^V2A2'‘  +B'A^-‘v2l»2+  J ®o'>o‘®2*2 

■ Ao'^®(,’‘o^“2*2'*<*2''2  +QA2^B2R‘‘b^V2  +V2A2-V2B2R‘‘b^V2)A2* 

= a - B R . 
o 000 


, A2^B2R'^B^A2‘^  = A^^B2HR’^H'B^A^’^  = A2^B2R^^B^A2' ^ and 


Furthermore 
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that  is, 

= -(V2 +A2'^Q2)A2^B2R’^B2V2A2^ 

= (V2  +A^'^Q)A2’'B2R'^B2A^‘^QA2^  +V2), 

implying  Xj^  ~ Pq  " elimination  of  Vj^  from  (20)  yields  the 

Hamilton-Jacobi  equation  (11)  of  the  reduced  problem. 
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